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WHAT IS CLAIMED IS: 

1. A vehicle tracking system for a vehicle, comprising: 
a first camera mounted on the vehicle; 

an image memory configured and arranged to store a first and second input images 
5 containing a remote vehicle received from the first camera; 

a vehicle detecting section configured and arranged to detect the remote vehicle 
based on the first input image stored in the image memory; 

a reference template creating section configured and arranged to extract an image 
region including the remote vehicle from the first input image to be used as a reference 
10 template; 

a reference template storing memory configured and arranged to store the reference 
template and a position in the first input image where the reference template was extracted; 

a vehicle tracking processing section configured and arranged to determine a 
position and an enlargement/reduction ratio for at least a portion of the second input image 
15 that provide a maximum correlation value between the portion of the second input image 
and the reference template with respect to the remote vehicle, while enlarging or reducing 
the portion of the second input image in consideration of a change in size of the remote 
vehicle in the second input image; 

a processing result outputting section configured and arranged to determine a 
20 position of the remote vehicle relative to the vehicle based on the position and the 

enlargement/reduction ratio of the portion of the second input image that provide the 
maximum correlation value; 

a vehicle image storage memory configured and arranged to consecutively store a 
vehicle image extracted from the portion of the second input image at the position and the 
25 enlargement/reduction ratio that provide the maximum correlation value; and 

a reference template updating section configured and arranged to normalize the 
vehicle image and the reference template, calculate a dispersion of at least one pixel of the 
reference template with respect to a pixel at a corresponding position in the vehicle image, 
and update the reference template by deleting the at least one pixel when the dispersion of 
30 the at least one pixel is equal to or greater than a threshold value. 
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2. The vehicle tracking system as recited in claim 1, further comprising 

a tracking reliability determining section configured and arranged to compare the 
maximum correlation value obtained by the vehicle tracking processing section with a 
threshold value to determine a reliability of vehicle tracking. 

5 

3. The vehicle tracking system as recited in claim 1, wherein 

the vehicle tracking processing section is further configured and arranged to detect 
edges of the remote vehicle in the first and second input images and determine the 
enlargement/reduction ratio to calculate the correlation based on change in distance 
10 between the edges in the first and second input images. 

4. The vehicle tracking system as recited in claim 1, further comprising 

a distance measurement section configured and arranged to obtain distance data 
indicative of a distance to the remote vehicle, 
15 the vehicle detecting section being further configured and arranged to detect the 

remote vehicle based on the distance data obtained by the distance measurement device, 
the reference template creating section being further configured and arranged to 
determine the image region including the remote vehicle based on detection of the remote 
vehicle by the vehicle detecting section, 
20 the reference template storage memory being further configured and arranged to 

store the distance data obtained when the first input image was captured, and 

the vehicle tracking processing section being further configured and arranged to 
determine the enlargement/reduction ratio based on a ratio of the distance data obtained 
when the second input image was captured to the distance data obtained when the first 
25 input image was captured. 

5. The vehicle tracking system as recited in claim 1, further comprising 
a second camera mounted on the vehicle; and 

a distance image creating section configured and arranged to determine a distance 
30 between the vehicle and the remote vehicle by creating a distance image including a 
distance value in each pixel obtained by calculating a parallax of the each pixel with 
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respect to the remote vehicle between the first input image and a third input image 
simultaneously obtained with the first input image by the second camera, 

the image memory being further configured and arranged to store the third input 

image, 

5 the reference template creating section being further configured and arranged such 

that extraction of the reference template from the image region is based on the distance 
image, 

the vehicle tracking processing section being further configured and arranged to 
determine the enlargement/reduction ratio based on a ratio of the distance when the second 
10 input image was captured to the distance when the first input image was captured, and 

the reference template updating section being further configured and arranged to 
delete at least one pixel that includes the dispersion that is equal to or greater than a 
threshold value and a distance value that is different from a distance value indicative of the 
remote vehicle. 

15 

6. The vehicle tracking system as recited in claim 1, wherein 

the image memory is further configured and arranged to obtain first and second 
differential images of the first and second input images, respectively; 

the reference template creating section is further configured and arranged to create 
20 the reference template using the differential image of the first input image; 

the vehicle tracking processing section is further configured and arranged to 
determine the enlargement/reduction ratio by using the first and second differential images 
to calculate the correlation value of the first and second input images. 

25 7. The vehicle tracking system as recited in claim 2, wherein 

the image memory is further configured and arranged to obtain first and second 
differential images of the first and second input images, respectively; 

the reference template creating section is further configured and arranged to create 
the reference template using the differential image of the first input image; 
30 the vehicle tracking processing section is further configured and arranged to 

determine the enlargement/reduction ratio by using the first and second differential images 
to calculate the correlation value of the first and second input images. 
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8. The vehicle tracking system as recited in claim 3, wherein 
the image memory is further configured and arranged to obtain first and second 

differential images of the first and second input images, respectively; 

the reference template creating section is further configured and arranged to create 
the reference template using the differential image of the first input image; 

the vehicle tracking processing section is further configured and arranged to 
determine the enlargement/reduction ratio by using the first and second differential images 
to calculate the correlation value of the first and second input images. 

9. The vehicle tracking system as recited in claim 4, wherein 
the image memory is further configured and arranged to obtain first and second 

differential images of the first and second input images, respectively; 

the reference template creating section is further configured and arranged to create 
the reference template using the differential image of the first input image; 

the vehicle tracking processing section is further configured and arranged to 
determine the enlargement/reduction ratio by using the first and second differential images 
to calculate the correlation value of the first and second input images. 

20 10. The vehicle tracking system as recited in claim 1, wherein 

the reference template updating section is further configured and arranged to 
prohibit deleting the at least one pixel when the dispersion of the at least one pixel is 
relatively small for a prescribed period. 

25 11. The vehicle tracking system as recited in claim 2, wherein 

the reference template updating section is further configured and arranged to 
prohibit deleting the at least one pixel when the dispersion of the at least one pixel is 
relatively small for a prescribed period. 
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12. The vehicle tracking system as recited in claim 3, wherein 

the reference template updating section is further configured and arranged to 
prohibit deleting the at least one pixel when the dispersion of the at least one pixel is 
relatively small for a prescribed period. 

13. The vehicle tracking system as recited in claim 4, wherein 

the reference template updating section is further configured and arranged to 
prohibit deleting the at least one pixel when the dispersion of the at least one pixel is 
relatively small for a prescribed period. 

14. The vehicle tracking system as recited in claim 1, wherein 

the reference template updating section is further configured and arranged to cease 
updating the reference template when a number of tracking processing cycles exceeds a 
prescribed number of cycles and the dispersion of all pixels of the reference template have 
been less than or equal to a prescribed value for a prescribed amount of time or longer. 

15. The vehicle tracking system as recited in claim 2, wherein 

the reference template updating section is further configured and arranged to cease 
updating the reference template when a number of tracking processing cycles exceeds a 
prescribed number of cycles and the dispersion of all pixels of the reference template have 
been less than or equal to a prescribed value for a prescribed amount of time or longer. 

16. The vehicle tracking system as recited in claim 3, wherein 

the reference template updating section is further configured and arranged to cease 
updating the reference template when a number of tracking processing cycles exceeds a 
prescribed number of cycles and the dispersion of all pixels of the reference template have 
been less than or equal to a prescribed value for a prescribed amount of time or longer. 

17. The vehicle tracking system as recited in claim 4, wherein 

the reference template updating section is further configured and arranged to cease 
updating the reference template when a number of tracking processing cycles exceeds a 
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prescribed number of cycles and the dispersion of all pixels of the reference template have 
been less than or equal to a prescribed value for a prescribed amount of time or longer, 

18. A method of tracking a remote vehicle by using image processing of a 
5 plurality of images photographed by a camera mounted on a main vehicle, comprising 

detecting the remote vehicle in a first input image; 

obtaining a reference template by extracting an image region containing the remote 
vehicle from the first input image when the remote vehicle is detected; 

tracking the remote vehicle by determining a position of a portion of a second 

10 input image containing the remote vehicle and an enlargement/reduction ratio of the 

portion of the second input image that provide a maximum correlation value between the 
reference template and the portion of the second input image, while enlarging or reducing 
the portion of the second input image in consideration of a change in size of the remote 
vehicle in the second input image; 

15 outputting a distance and a direction of the remote vehicle with respect to the 

vehicle as a remote vehicle detection result determined based on the position and the 
enlargement/reduction ratio of the second image at which the maximum correlation value 
was obtained; 

storing at least one vehicle image extracted from the second image at the position 
20 and the enlargement/reduction ratio for which the maximum correlation value was 
obtained; 

normalizing the reference template and the at least one vehicle image stored 
consecutively; 

calculating a dispersion of at least one pixel in the reference template by 
25 comparing the reference template and the at least one vehicle image at a pixel that 
corresponds to the at least one pixel in the reference template; 

updating the reference template by consecutively deleting the at least one pixel 
when the dispersion of the at least one pixel is equal to or greater than a threshold value to 
obtain an updated reference template; and 
30 using the updated reference template to execute the tracking of the remote vehicle. 
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19. A vehicle tracking system for a vehicle, comprising 

image inputting means for inputting a first and second input images; 

image storing means for storing the first and second input images obtained in the 

image inputting means; 
5 remote vehicle detecting means for detecting a remote vehicle in the first input 

image; 

reference template creating means for creating a reference template including the 
remote vehicle extracted from the first input image based on a detection result in the 
remote vehicle detecting means; 
10 vehicle tracking means for tracking the remote vehicle by calculating a position 

and an enlargement/reduction ratio of a portion of the second input image that provide a 
maximum correlation between the reference template and the second input image with 
respect to the remote vehicle; 

tracking result outputting means for determining and outputting the position and 
15 the enlargement/reduction ratio of the portion of the second input image that provide the 
maximum correlation; 

vehicle image storing means for storing a vehicle image extracted from the second 
input image at the position and the enlargement/reduction ratio that provide the maximum 
correlation; and 

20 reference template updating means for updating the reference template by 

normalizing the vehicle image and the reference template, calculating a dispersion of at 
least one pixel of the reference template with respect to a pixel at a corresponding position 
in the vehicle image, and deleting the at least one pixel of the reference template when the 
dispersion of the at least one pixel is equal to or greater than a threshold value. 

25 
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